The invention relates generally to the field of image capture, and more specifically to a method of removing noise from a sparsely sampled extended dynamic range digital image.
Image sensing devices, such as a charge-coupled device (CCD) and CMOS image sensors, are commonly found in such products as digital cameras, scanners, and video cameras. These image sensing devices have a limited dynamic range when compared to traditional photographic film products. A typical electronic image sensing device has a dynamic range of about 7 stops. This means that the exposure for a typical scene must be determined with a fair amount of accuracy in order to avoid clipping the resultant signal. By contrast, natural scenes often exhibit a dynamic range of 9 stops and higher. This is mainly a consequence of multiple light sources with widely varying intensities illuminating the scene objects. Specular highlights also contribute to the dynamic range of natural scenes.
Electronic sensors used to scan photographic film must also contend with a high dynamic range of signal intensities. In U.S. Pat. No. 5,221,848 issued Jun. 22, 1993 to Milch entitled High Dynamic Range Film Digitizer and Method of Operating the Same discloses a method and apparatus designed to extend the dynamic range of an electronic image sensor. Aimed primarily for scanning photographic film, Milch teaches a method of a one pass film scanner using a charge-coupled device scanner having a plurality of linear arrays thereon. One of the arrays is responsive to high intensities of light and the other array is responsive to low intensities of light. The information from the two arrays is then combined and digitized forming an extended dynamic range digital image. The method and apparatus disclosed by Milch is an electronic image sensor having photosites with the same spectral sensitivity but different inherent response to intensities of light which is capable of producing digital images with very high dynamic range.
The noise present in signals produced by electronic image sensing devices can be removed with the application of a noise reduction algorithm. An example of noise reduction algorithm is the Sigma Filter, described by Jong-Sen Lee in the journal article xe2x80x9cDigital Image Smoothing and the Sigma Filterxe2x80x9d, Computer Vision, Graphics, and Image Processing, Vol. 24, 1983, pp. 255-269. Lee discloses a noise reduction filter that uses a non-linear pixel averaging technique sampled from a rectangular window about the center pixel. Pixels in the local neighborhood are either included or excluded from the numerical average on the basis of the difference between the pixel and the center pixel. The Sigma Filter was designed for image processing applications for which the dominant noise source is Gaussian additive noise. Signal dependent noise sources can be incorporated by making noise reduction control parameters a function of the signal strength. However, for both signal independent and signal dependent noise cases the expected noise standard deviation must be known to obtain optimal results.
The Sigma noise reduction method described by Lee cannot be optimally applied directly to the signals produced by extended dynamic range electronic image sensing devices due to the fact that the noise characteristics of the different types of photosites have different noise characteristics. Therefore, there exists a need for an improved method of noise reduction for images produced by electronic image sensors capable of recording images with extended dynamic range.
The need is met according to the present invention by providing a method of removing noise from a sparsely sampled extended dynamic range image produced by a sparsely sampled extended dynamic range image sensing device having fast photosites with a predetermined response to a light exposure for producing fast pixel values and slow photosites with a slower response to the same light exposure for producing slow pixel values, that includes the steps of: providing a sparsely sampled extended dynamic range digital image having fast and slow pixel values; using a noise filter method to remove noise from the sparsely sampled extended dynamic range digital image by using only slow pixel values to generate slow noise reduced pixel values and using only fast pixel values to generate fast noise reduced pixel values; and combining the slow noise reduced pixel values and the fast noise reduced pixel values to generate a noise reduced sparsely sampled digital image.
An important advantage of the present invention is the employment of a noise reduction method optimized for each type pixel relating to the different types of photosites employed by an extended dynamic range image sensing device.
Another important advantage of the present invention relates to the sequence of employing a noise removal processing step before employing an interpolation processing step. The processed digital images produced by the present invention have less noise than if the sequence of employing these two processing steps were reversed.
Still another important advantage of the present invention is the incorporation of a signal dependent noise reduction method which is used to optimize the noise reduction method for pixels of the image signal with different noise characteristics.